1. Field of the Invention
The present invention relates to a method of compensating for the overall brightness variation between image frames required for image processing such as in video editing and coding for efficient transmission and storage of digital video images.
2. Background Art
As a conventional coding technique for efficiently transmitting and storing video image data, a technique known as motion compensation interframe prediction coding has been widely used. With this technique, the luminance values (or color differences) of the image frames being processed are not directly transmitted and stored; instead, the technique measures how much objects inside an image have moved between a video image frame which has already been coded (referred to as a reference image frame) and the image frame being processed, and luminance difference values at positions which are shifted by the distance of movement are transmitted and stored.
Movement compensation interframe prediction coding enables transmission and storage of reduced quantities of data because the difference values are generally less than the luminance values themselves.
In this case, when measuring movement, it is usually assumed that the image frame being processed and the reference image frame have the same lighting conditions. That is, movement is detected under the assumption that corresponding objects in both frames have the same luminance values. However, the luminance values for corresponding objects can differ considerably between frames in cases such as when there is flickering, when a strobe light has flashed, or when the entire image is made gradually brighter (hereinafter referred to as xe2x80x9cfade-inxe2x80x9d) or darker (hereinafter referred to as xe2x80x9cfade-outxe2x80x9d) by adjusting the diaphragm of the camera or by using video editing equipment. Consequently, the following problems are associated with motion compensation interframe prediction coding:
(1) The detection of movement is not properly performed.
(2) The coding efficiency is not sufficiently improved.
As a conventional method principally for resolving the above-mentioned problem (1), there is a technique of measuring luminance changes between image frames, and performing movement measurements while correcting for the measured luminance changes.
Luminance changes can be largely separated into types wherein the degree of change changes according to the spatial position in the image frame, such as those which are caused by noise, and types wherein the degree of change is the same over the entire image frame, such as those which are caused by fade-ins and fade-outs.
Since it is important to measure the luminance changes as accurately as possible when attempting to resolve problem (1), the measurements are usually made in small areas within the frame in order to handle both types of luminance changes. However, although this resolves problem (1), the size of the interframe luminance difference values can still result in problem (2).
Thus, there is also a technique of reducing the interframe luminance difference values by transmitting and storing the luminance changes of the small areas as coded data, and correcting for these luminance changes at the decoder side. However, since this technique involves sending coded data expressing the luminance change for each small area, the quantities of data are ultimately increased to such a degree that the overall data quantity is not much different. In other words, the coding efficiency is not much improved.
On the other hand, there is a technique of measuring a single luminance change for the image frame as a whole, and transmitting and storing that luminance change as coded data for the purposes of improving the coding efficiency when the same luminance change has occurred over the entire image frame. With this technique, the quantity of coded data is extremely small because it is sufficient to send only a single piece of coded data expressing the luminance change for each image frame. Additionally, this technique contributes largely to the improvement of coding efficiency in cases of fade-ins and fade-outs because the interframe luminance change values can be made somewhat smaller simply by correcting for the luminance change.
As a specific parameter for expressing the overall luminance change, this technique uses only a parameter expressing the gain difference. That is, taking a parameter DB representing the gain difference, it is assumed that the luminance value x of each pixel will change to xxe2x80x2 as shown in Equation 1:
xxe2x80x2=x+DBxe2x80x83xe2x80x83(1) 
In this case, the luminance change x is always constant:
xcex94x=xxe2x80x2xe2x88x92x=DBxe2x80x83xe2x80x83(2) 
FIG. 3 shows the changes in the luminance values in an actual case wherein a still image containing various brightness values was taken by a camera and the diaphragm of the camera was gradually closed so as to darken the overall image, i.e. a fade-out. When considering the changes in the luminance values from frame 1 to frame 11, or frame 11 to frame 21, the luminosity changes are approximately constant regardless of the luminance values themselves. Therefore, in this case, the assumption of Equation (1) holds. That is, the constant amount of change corresponds to DB in Equation (1).
On the other hand, FIG. 4 shows the changes in the luminance values in a case wherein the same still image was taken without changing the diaphragm of the camera, then a fade-out was caused using digital editing equipment. When considering the changes in the luminance values from frame 41 to frame 51, or frame 51 to frame 61, the luminance changes change depending on the luminance values. Therefore, the assumption of Equation (1) does not hold in this case, and the precision is insufficient to express overall luminance changes using only the parameter DB.
When the images are divided into small areas of 8xc3x978 pixels, the variations in the luminances of these small areas are generally not very large. For example, when the luminance value in the image frame overall changes from 0-255, the variations in the small areas are almost always in the range of 20-40. In this range, the luminance change can be considered to be approximately constant without regard to the luminance values themselves. That is, the assumption of Equation (1) is still appropriate if the luminance change is found only in small areas.
However, the range of luminance values broadens when the image frame is taken as a whole. That is, there is a considerable possibility that there will be luminance changes covering the entire range of luminance values 0-255. Therefore, the approximation of Equation (1) is insufficient for determining one type of luminance change over the entire image frame.
The object of the present invention is to offer a brightness variation compensation method and coding/decoding apparatus for moving images, capable of more efficiently coding moving images with overall brightness variations.
Therefore, the present invention offers a moving image brightness variation compensation method for encoding digital moving images for transmission and storage, and for image processing when editing moving images, the moving image brightness variation compensation method comprising a step of compensating for overall brightness variations by correcting a luminance value x of each pixel according to the formula DCxc2x7x+DB, wherein DB is a parameter indicating a gain change and DC is a parameter indicating a contrast change, the parameters representing overall luminance changes between a reference image plane and an image plane being processed.
According to this method, coding procedures can be made more efficient when coding moving images having overall brightness variations, due to the use of a parameter indicating the contrast change in addition to a parameter indicating the gain change, when measuring or compensating for overall brightness variations.
Additionally, the present invention offers a moving image coding apparatus, comprising parameter determining means for determining a parameter DB indicating a gain change and a parameter DC indicating a contrast change, the parameters representing overall luminance changes between a reference image plane and an image plane being processed; and brightness variation compensation means for correcting a luminance value x of each pixel in the image plane being processed using the parameters DB and DC according to the formula DCxc2x7x+DB.
According to this apparatus, coding procedures can be made more efficient when coding moving images having overall brightness variations, due to the use of a parameter indicating the contrast change in addition to a parameter indicating the gain change, when measuring or compensating for overall brightness variations.
Additionally, the present invention offers a computer-readable recording medium recording a moving image coding program, the moving image coding program containing instructions for a computer to execute a parameter determining step of determining a parameter DB indicating a gain change and a parameter DC indicating a contrast change, the parameters representing overall luminance changes between a reference image plane and an image plane being processed; and a brightness variation compensation step of correcting a luminance value x of each pixel in the image plane being processed using the parameters DB and DC according to the formula DCxc2x7x+DB.
By reading the program recorded onto this recording medium and executing the program with a computer, coding procedures can be made more efficient when coding moving images having overall brightness variations, due to the use of a parameter indicating the contrast change in addition to a parameter indicating the gain change, when measuring or compensating for overall brightness variations.
Additionally, the present invention offers a computer-readable recording medium recording coded data of moving images, wherein the coded data includes a plurality of areas recording image plane coded data encoded by units of image planes, the areas recording image plane coded data comprising a parameter area recording parameters DB indicating gain changes and DC indicating contrast changes, the parameters representing overall luminance changes between reference image planes and image planes being processed; and a coded area recording data in units of image planes relating to differences in pixel values for corresponding pixels between the reference image planes and image planes wherein luminance values x of the pixels in the reference image planes have been corrected by using the parameters DB and DC according to the formula DCxc2x7x+DB.
Since the coded data on this recording medium includes parameters indicating the contrast change in addition to parameters indicating the gain change when measuring or compensating for overall brightness variations, and includes data coded by using these parameters, coding procedures can be made more efficient when coding moving images having overall brightness variations.